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27 May 1976 


MEMORANDUM FOR: 

DC/PAID/ICS 

SUBJECT: 27 April 1976 Draft Study Entitled 

"The Intelligence Community's Performance 
Against the Key Intelligence Questions 
for FY 1975 


1. The comments below reflect my views and those of 
my colleagues in the NIO structure. We all appreciate that 
you were saddled by the previous DCI with a virtually . 
impossible task. With impossible tasks, however, it is 
difficult to produce satisfactory solutions. A lot o± 
effort clearly went into this study but it itself con- 
vinces me that we are not yet near the point where we 
can "measure the Community's performance the way Colby 
wanted it measured — i.e., develop meaningful numerical 
data which provide a hard index of the cost effectiveness 
or even the relative utility of the output of the Intel 
ligence Community's various components. 


2. Enough hard work has gone into this problem so 
that we can now say that we have made an honest try. 

From this try, however, it is clear that the quantitative 
indicators now available to describe the distribuiton of 
effort throughout the Community and the attribution of 
costs -- "objectives," "commitments," and CIRIS program 
data -- are simply not adequate in concept or content 
to be of much real value in illuminating resource 
allocation options or decisions. Colby wanted us to 
square the circle. We have, in effect, shown what ^ 
mathematicians have long known, namely, that the circle 
cannot be squared. 


3. I would strongly re 
be submitted to the NFIB but 
to the drawing board and pos 
CFI and the DCI the basic qu 
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process — the production of intelligence -- whoS ® 
will inevitably be assessed in subjective terms, for the 
line ./the utility of Intelligence has to be the 
policy-level user's determination of the extent t 
it helped him understand the decisions he has to ma . 

That, I submit, is a kind of judgment which 

fication. Furthermore, the intelligence pro uc ± t _ 

process by which it is produced involve a host J 

cost and joint-product complexities which are ^ ost 
impossible to disentangle without artificial, arbitr y 

oversimplification. 

4. Clearly the Community needs an evaluation process, 
and clearly it would be ideal if that process could produce 

data which virtually dictate resource deC ^ n ^ ad W ^ s we 
certainly should try to move as far down this road as we 
can but we should be under no illusion about our ability 
to reach its end. What we need, in the NIOs opinion 
not further reworking of what looks more and more like a 
blind alley but a fresh look at where we want to go and 
what ^ seems 7 the most sensible route to follow in getting 

there . 

5 The above comments may be regarded as fighting 
the problem. Perhaps they are. If the decision is made 
iSlt Ihls (or some) version of a FY 1975 KIQ Evaluation 
must be presented to the NFIB, we »»"“ '"'f St , ", 
cosmetic modifications to the attached draft, 
its title should be changed to something like An 
Alternative Attempt to Measure the Intelligence Community s 
Performance Against the Key Intelligence Questions for 
FY 1975." Second, if this draft is to be circulated to the 
NFIB it ought to be distributed in something akin to its 
current^" format . In other words, the document circulated 
should clearly be a draft, offered as such. It should 

not appear with a flossy cover or m a P in ... this 

that looks like an end product, as was the case with this 
study /predecessor. No matter what caveats are put in 
tie level note about the study's being an «J'“““tal 
approach offered for consideration, a slick prin 
version will look to the NFIB members like a finished 
product they have to accept or reject virtually 
entirety. 
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6. On a more substantive point, we believe that Section I 
and the Figures fall somewhat into the same trap as the earlier 
evaluation, by aggregating basically meaningless statistical 
costing and performance data (described in the Forward as 
incomplete and in other places as lacking validity) and 
narrative comments by Substantive Objectives. The 
aggregated quantitative production and collection data 
contained in pages 5 through 7 regarding numbers of ob- 
jectives (473) and commitments (697), and the unexplained 
CIRIS cost data of page 9 are unspecified, undefined, 
admittedly unreliable, and, therefore, misleading. The 
aggregative approach was bitterly attacked during the 
discussion of the earlier att emp t by the USIB members 

who were more interested in a subjective KIQ-by-KIQ 
evaluation . 

7. Along this line, the Annex appears to come 
closer to the type of information in which we believe 
the USIB (now NFIB) members are interested and may find 

of value. However, the "unreliable" costing data are also 
reflected in the Annex in the Investment and Return 
section of the evaluations of the individual KIQ's. In 
view of the admittedly questionable utility of the 
available costing data, it would appear that these 
sections of the Annex really raise more problems than 
they solve, while the other sections reflect the 
desiderata expressed by the then USIB members at their 
discussion of the first evaluation attempt. In sum, I 
feel that a reduction/deletion of some of the statistical 
data in Section I and the Annex would make this a more 
worthwhile paper for consideration and discussion by the 
NFIB membership if it has to be presented to the NFIB. 
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Geo rge Jk . Carver , Jr. 

Deputy for National Intelligence Officers 

GACarver, Jr. /If 
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